<!DOCTYPE html>
<html class="writer-html5" lang="Python" >
<head>
  <meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Continuous Tabular Benchmarking module &mdash; Salesforce CausalAI Library 1.0 documentation</title>
      <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
      <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
  <!--[if lt IE 9]>
    <script src="_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
        <script src="_static/jquery.js"></script>
        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
        <script src="_static/doctools.js"></script>
        <script src="_static/sphinx_highlight.js"></script>
        <script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
    <script src="_static/js/theme.js"></script>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" /> 
</head>

<body class="wy-body-for-nav"> 
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >

          
          
          <a href="index.html" class="icon icon-home">
            Salesforce CausalAI Library
          </a>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>
        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Prior%20Knowledge.html">Prior Knowledge</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Data%20objects.html">Data Object</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Data%20Generator.html">Data Generator</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/PC_Algorithm_TimeSeries.html">PC algorithm for time series causal discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/GrangerAlgorithm_TimeSeries.html">Ganger Causality for Time Series Causal Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/VARLINGAM_Algorithm_TimeSeries.html">VARLINGAM for Time Series Causal Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/PC_Algorithm_Tabular.html">PC Algorithm for Tabular Causal Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/GES_Algorithm_Tabular.html">GES for Tabular Causal Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/LINGAM_Algorithm_Tabular.html">LINGAM for Tabular Causal Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/GIN_Algorithm_Tabular.html">Generalized Independent Noise (GIN)</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/GrowShrink_Algorithm_Tabular.html">Grow-Shrink Algorithm for Tabular Markov Blanket Discovery</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Benchmarking%20Tabular.html">Benchmark Tabular Causal Discovery Algorithms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Benchmarking%20TimeSeries.html">Benchmark Time Series Causal Discovery Algorithms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Causal%20Inference%20Time%20Series%20Data.html">Causal Inference for Time Series</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="tutorials/Causal%20Inference%20Tabular%20Data.html">Causal Inference for Tabular Data</a></li>
</ul>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="index.html">Salesforce CausalAI Library</a>
      </nav>

      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
      <li class="breadcrumb-item active">Continuous Tabular Benchmarking module</li>
      <li class="wy-breadcrumbs-aside">
            <a href="_sources/benchmark.tabular.continuous.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
             
  <section id="module-causalai.benchmark.tabular">
<span id="continuous-tabular-benchmarking-module"></span><h1>Continuous Tabular Benchmarking module<a class="headerlink" href="#module-causalai.benchmark.tabular" title="Permalink to this heading"></a></h1>
<section id="module-causalai.benchmark.tabular.continuous">
<span id="causalai-benchmark-tabular-continuous"></span><h2>causalai.benchmark.tabular.continuous<a class="headerlink" href="#module-causalai.benchmark.tabular.continuous" title="Permalink to this heading"></a></h2>
<p>This is the benchmarking module for continuous tabular data. This module supports methods that evaluates causal discovery algorithms against various challenges, 
such as their sample complexity, variable complexity, etc. Users can use either synthetically generated data, or provide their own data for benchmarking.</p>
<p>The default evaluation metrics supported by this module are Precision, Recall, F1 Score, and Time Taken by the algorithm. There is also an option for users to 
include their own custom metrics when calling the benchmarking module.</p>
<p>We provide support for a default set of causal discovery algorithms. Users also have the option to include their own algorithm when calling the 
benchmarking module.</p>
<p>Data:</p>
<p>1. Synthetic data: this module randomly generates both the causal graph (and the corresponding structural equation model) and the data associated 
with it. This module supports several benchmarking methods which evaluate causasl discovery algorithms on various aspects such as sample complexity, 
variable complexity, graph sparsity, etc. Depending on what is being evaluated, the corresponding method generates the graphs and data accordingly. 
Synthetic data evaluation serves two purposes:</p>
<blockquote>
<div><ol class="loweralpha simple">
<li><p>compare the performance of each causal discovery algorithm across different values of a variant (e.g. increasing number of sample),</p></li>
<li><p>compare the performance of different causal discovery algorithms for any on given value of a variant.</p></li>
</ol>
</div></blockquote>
<p>2. User provided data: In this case, since the data is fixed, this module helps evaluate the performance of one or more causal discovery algorithms 
on user provided data. Since the data is not synthetically generated, in order to compute the evaluation metrics such as Precision/Recall, we need the 
ground truth causal graph. Therefore, the user provided data accepted by this module must contain this information. Specifically, the data must be a list 
of tuples, where each tuple contains the triplet (data_array, var_names, graph_gt), where data_array is a 2D Numpy data array of shape (samples x variables), 
var_names is a list of variable names, and graph_gt is the ground truth causal graph in the form of a Python dictionary, where keys are the variable names, 
and the corresponding values are a list of parent names.</p>
<dl class="py class">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">causalai.benchmark.tabular.continuous.</span></span><span class="sig-name descname"><span class="pre">BenchmarkContinuousTabular</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">algo_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">kargs_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_exp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">20</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">custom_metric_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular" title="Permalink to this definition"></a></dt>
<dd><p>Continuous tabular data benchmarking module. This class inherits the methods and 
variables from BenchmarkTabularBase and BenchmarkContinuousTabularBase, and defines 
benchmarking methods that evaluates causal discovery algorithms against various challenges, 
such as their sample complexity, variable complexity, etc.</p>
<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.__init__">
<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">algo_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">kargs_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_exp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">20</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">custom_metric_dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">None</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.__init__" title="Permalink to this definition"></a></dt>
<dd><p>Continuous tabular data benchmarking module</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>algo_dict</strong> (<em>Dict</em>) -- <p>A Python dictionary where keys are names of causal discovery algorithms, and 
values are the unistantiated class objects for the corresponding algorithm. Note that this class 
must be inherited from the <cite>BaseTabularAlgoFull</cite> class that can be found in causalai.models.tabular.base.
Crucially, this class constructor must take a <cite>TabularData</cite> object (found in causalai.data.tabular) as input, 
and should have a <cite>run</cite> method which performs the causal discovery and returns a Python dictionary. The keys of this 
dictionary should be of the form:</p>
<dl class="simple">
<dt>{</dt><dd><p>var_name1: {'parents': [par(var_name1)]},
var_name2: {'parents': [par(var_name2)]}</p>
</dd>
</dl>
<p>}</p>
<p>where par(.) denotes the parent variable name of the argument variable name.</p>
</p></li>
<li><p><strong>kargs_dict</strong> (<em>Dict</em>) -- A Python dictionary where keys are names of causal discovery algorithms (same as algo_dict), 
and the corresponding values contain any arguments to be passed to the <cite>run</cite> method of the class object specified in 
algo_dict.</p></li>
<li><p><strong>num_exp</strong> (<em>int</em>) -- The number of independent runs to perform per experiment, each with a different random seed. A different 
random seed generates a different synthetic graph and data for any given configuration. Note that for use provided data, 
num_exp is not used.</p></li>
<li><p><strong>custom_metric_dict</strong> (<em>Dict</em>) -- A Python dictionary for specifying custom metrics in addition to the default evaluation metrics 
calculated for each experiment (precision, recall, F1 score, and time taken). The keys of this dictionary are the names 
of the user specified metrics, and the corresponding values are callable functions that take as input (graph_est, graph_gt). 
Here graph_est and graph_gt are the estimated and ground truth causal graph. These graphs are specified as Python Dictionaries, 
where keys are the children names, and the corresponding values are lists of parent variable names.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_graph_density">
<span class="sig-name descname"><span class="pre">benchmark_graph_density</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="pre">graph_density_list:</span> <span class="pre">~typing.List[float]</span> <span class="pre">=</span> <span class="pre">[0.05,</span> <span class="pre">0.1,</span> <span class="pre">0.2,</span> <span class="pre">0.5],</span> <span class="pre">num_vars:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">20,</span> <span class="pre">T:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">1000,</span> <span class="pre">fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;function</span> <span class="pre">BenchmarkContinuousTabular.&lt;lambda&gt;&gt;,</span> <span class="pre">coef:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">noise_fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;built-in</span> <span class="pre">method</span> <span class="pre">randn</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;</span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_graph_density" title="Permalink to this definition"></a></dt>
<dd><p>Graph density: Benchmark algorithms on synthetic data with different number of samples. 
The synthetic data for any variable is generated using a structural equation model (SEM) of the form:</p>
<p>child = sum_i coef* parent_i + noise</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>graph_density_list</strong> (<em>List</em><em>[</em><em>float</em><em>]</em>) -- It contains list of graph denity values to be used to generate the causal graph. 
Each value must be in (0,1].</p></li>
<li><p><strong>num_vars</strong> (<em>int</em>) -- Integer value specifying the number of variables in the generated data.</p></li>
<li><p><strong>T</strong> (<em>int</em>) -- Integer value specifying the number of samples in the generated data.</p></li>
<li><p><strong>fn</strong> (<em>Callable</em>) -- Callable function that acts as the non-linearity on parent variable value in the structural 
equation model. This same function is applied on all parents.</p></li>
<li><p><strong>coef</strong> (<em>float</em>) -- Coefficient for the parent variable value in the structural 
equation model. This same coefficient is used for all the parents.</p></li>
<li><p><strong>noise_fn</strong> (<em>Callable</em>) -- Callable function from which noise is sampled in the structural 
equation model. This same function is used in all the equations.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_noise_type">
<span class="sig-name descname"><span class="pre">benchmark_noise_type</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="pre">noise_fn_types:</span> <span class="pre">~typing.List[~typing.Callable]</span> <span class="pre">=</span> <span class="pre">[&lt;built-in</span> <span class="pre">method</span> <span class="pre">randn</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;,</span> <span class="pre">&lt;built-in</span> <span class="pre">method</span> <span class="pre">rand</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;],</span> <span class="pre">noise_fn_names:</span> <span class="pre">~typing.List[str]</span> <span class="pre">=</span> <span class="pre">['Gaussian',</span> <span class="pre">'Uniform'],</span> <span class="pre">num_vars:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">20,</span> <span class="pre">graph_density:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">T:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">1000,</span> <span class="pre">fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;function</span> <span class="pre">BenchmarkContinuousTabular.&lt;lambda&gt;&gt;,</span> <span class="pre">coef:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1</span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_noise_type" title="Permalink to this definition"></a></dt>
<dd><p>Continuous noise type: Benchmark algorithms on synthetic data generated using different noise functions. 
The synthetic data for any variable is generated using a structural equation model (SEM) of the form:</p>
<p>child = sum_i coef* parent_i + noise</p>
<p>Here noise is sampled from one of the provided noise_fn_types.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>noise_fn_types</strong> (<em>List</em><em>[</em><em>Callable</em><em>]</em>) -- A list of callable functions, which when called, returns a scalar value. E.g. numpy.random.randn.</p></li>
<li><p><strong>noise_fn_names</strong> (<em>List</em><em>[</em><em>str</em><em>]</em>) -- A list of noise function names (string) corresponding to the list of noise functions
provided in noise_fn_types.</p></li>
<li><p><strong>num_vars</strong> (<em>int</em>) -- Integer value specifying the number of variables in the generated data.</p></li>
<li><p><strong>graph_density</strong> (<em>float</em>) -- Float value in (0,1] specifying the density of the causal graph. The value is used 
as the probability with which an edge between 2 nodes exists during the causal graph generation process.</p></li>
<li><p><strong>T</strong> (<em>int</em>) -- Integer value specifying the number of samples in the generated data.</p></li>
<li><p><strong>fn</strong> (<em>Callable</em>) -- Callable function that acts as the non-linearity on parent variable value in the structural 
equation model. This same function is applied on all parents.</p></li>
<li><p><strong>coef</strong> (<em>float</em>) -- Coefficient for the parent variable value in the structural 
equation model. This same coefficient is used for all the parents.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_sample_complexity">
<span class="sig-name descname"><span class="pre">benchmark_sample_complexity</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="pre">T_list:</span> <span class="pre">~typing.List[int]</span> <span class="pre">=</span> <span class="pre">[100,</span> <span class="pre">500,</span> <span class="pre">1000,</span> <span class="pre">5000],</span> <span class="pre">num_vars:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">20,</span> <span class="pre">graph_density:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;function</span> <span class="pre">BenchmarkContinuousTabular.&lt;lambda&gt;&gt;,</span> <span class="pre">coef:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">noise_fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;built-in</span> <span class="pre">method</span> <span class="pre">randn</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;</span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_sample_complexity" title="Permalink to this definition"></a></dt>
<dd><p>Sample Complexity: Benchmark algorithms on synthetic data with different number of samples. 
The synthetic data for any variable is generated using a structural equation model (SEM) of the form:</p>
<p>child = sum_i coef* parent_i + noise</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>T_list</strong> (<em>List</em><em>[</em><em>int</em><em>]</em>) -- It contains list of number of samples to be used to generate synthetic data.</p></li>
<li><p><strong>num_vars</strong> (<em>int</em>) -- Integer value specifying the number of variables in the generated data.</p></li>
<li><p><strong>graph_density</strong> (<em>float</em>) -- Float value in (0,1] specifying the density of the causal graph. The value is used 
as the probability with which an edge between 2 nodes exists during the causal graph generation process.</p></li>
<li><p><strong>fn</strong> (<em>Callable</em>) -- Callable function that acts as the non-linearity on parent variable value in the structural 
equation model. This same function is applied on all parents.</p></li>
<li><p><strong>coef</strong> (<em>float</em>) -- Coefficient for the parent variable value in the structural 
equation model. This same coefficient is used for all the parents.</p></li>
<li><p><strong>noise_fn</strong> (<em>Callable</em>) -- Callable function from which noise is sampled in the structural 
equation model. This same function is used in all the equations.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_snr">
<span class="sig-name descname"><span class="pre">benchmark_snr</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="pre">snr_list:</span> <span class="pre">~typing.List[float]</span> <span class="pre">=</span> <span class="pre">[0.01,</span> <span class="pre">0.1,</span> <span class="pre">1.0],</span> <span class="pre">num_vars:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">20,</span> <span class="pre">graph_density:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">T:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">1000,</span> <span class="pre">fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;function</span> <span class="pre">BenchmarkContinuousTabular.&lt;lambda&gt;&gt;,</span> <span class="pre">noise_fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;built-in</span> <span class="pre">method</span> <span class="pre">randn</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;</span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_snr" title="Permalink to this definition"></a></dt>
<dd><p>Signal to noise ratio: Benchmark algorithms on synthetic data generated with different signal to noise ratios. 
The synthetic data for any variable is generated using a structural equation model (SEM) of the form:</p>
<p>child = sum_i coef* parent_i + noise</p>
<p>We vary SNR by varying the value of coef above. Since the scale of noise is fixed, the ratio of coef to the scale
of noise acts as SNR.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>snr_list</strong> (<em>List</em><em>[</em><em>float</em><em>]</em>) -- It contains list of graph denity values to be used to generate the causal graph. 
Each value must be in (0,1].</p></li>
<li><p><strong>num_vars</strong> (<em>int</em>) -- Integer value specifying the number of variables in the generated data.</p></li>
<li><p><strong>graph_density</strong> (<em>float</em>) -- Float value in (0,1] specifying the density of the causal graph. The value is used 
as the probability with which an edge between 2 nodes exists during the causal graph generation process.</p></li>
<li><p><strong>T</strong> (<em>int</em>) -- Integer value specifying the number of samples in the generated data.</p></li>
<li><p><strong>fn</strong> (<em>Callable</em>) -- Callable function that acts as the non-linearity on parent variable value in the structural 
equation model. This same function is applied on all parents.</p></li>
<li><p><strong>noise_fn</strong> (<em>Callable</em>) -- Callable function from which noise is sampled in the structural 
equation model. This same function is used in all the equations.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="py method">
<dt class="sig sig-object py" id="causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_variable_complexity">
<span class="sig-name descname"><span class="pre">benchmark_variable_complexity</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="pre">num_vars_list:</span> <span class="pre">~typing.List[int]</span> <span class="pre">=</span> <span class="pre">[2,</span> <span class="pre">10,</span> <span class="pre">20,</span> <span class="pre">40],</span> <span class="pre">graph_density:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">T:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">1000,</span> <span class="pre">fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;function</span> <span class="pre">BenchmarkContinuousTabular.&lt;lambda&gt;&gt;,</span> <span class="pre">coef:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.1,</span> <span class="pre">noise_fn:</span> <span class="pre">~typing.Callable</span> <span class="pre">=</span> <span class="pre">&lt;built-in</span> <span class="pre">method</span> <span class="pre">randn</span> <span class="pre">of</span> <span class="pre">numpy.random.mtrand.RandomState</span> <span class="pre">object&gt;</span></em><span class="sig-paren">)</span><a class="headerlink" href="#causalai.benchmark.tabular.continuous.BenchmarkContinuousTabular.benchmark_variable_complexity" title="Permalink to this definition"></a></dt>
<dd><p>Variable Complexity: Benchmark algorithms on synthetic data with different number of variables. 
The synthetic data for any variable is generated using a structural equation model (SEM) of the form:</p>
<p>child = sum_i coef* parent_i + noise</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>num_vars_list</strong> (<em>List</em><em>[</em><em>int</em><em>]</em>) -- It contains list of number of variables to be used to generate synthetic data.</p></li>
<li><p><strong>graph_density</strong> (<em>float</em>) -- Float value in (0,1] specifying the density of the causal graph. The value is used 
as the probability with which an edge between 2 nodes exists during the causal graph generation process.</p></li>
<li><p><strong>T</strong> (<em>int</em>) -- Integer value specifying the number of samples in the generated data.</p></li>
<li><p><strong>fn</strong> (<em>Callable</em>) -- Callable function that acts as the non-linearity on parent variable value in the structural 
equation model. This same function is applied on all parents.</p></li>
<li><p><strong>coef</strong> (<em>float</em>) -- Coefficient for the parent variable value in the structural 
equation model. This same coefficient is used for all the parents.</p></li>
<li><p><strong>noise_fn</strong> (<em>Callable</em>) -- Callable function from which noise is sampled in the structural 
equation model. This same function is used in all the equations.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

</dd></dl>

</section>
</section>


           </div>
          </div>
          <footer>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2022, salesforce.com, inc..</p>
  </div>

  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
    provided by <a href="https://readthedocs.org">Read the Docs</a>.
   

</footer>
        </div>
      </div>
    </section>
  </div>
  <script>
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script> 

</body>
</html>